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Abstract 

Introduction. In recent years, the introduction of social question and answer services and other Internet tools 
have expanded the ways in which people have their questions answered. There has been speculation and debate 
over whether such services and other Internet tools are replacing library virtual reference services. 

Method. Most previous research on the comparison between social question and answer services and virtual 
reference has been conducted from the perspectives of libraries or library users, and research based on social 
question and answer services users' viewpoints has not yet been explored. This study surveyed Yahoo! Answers 
users on their awareness, use, and evaluation of the social question and answer services site and virtual 
reference. 

Analysis. The survey data from multiple-choice questions, fill-in, and open-ended questions were analysed 
using SPSS for a quantitative summary. 

Results. The findings of this study show that social question and answer services and virtual reference have 
their own strengths and weaknesses in serving various user needs and in meeting the expectations of an online 
or virtual service, based on important user-identified considerations. 

Conclusions. The notion that social question and answer services will take over virtual reference is not 
supported. As they stand, virtual reference and social question and answer services have great potential for 
collaboration in meeting user information needs. The practical implications of the findings to virtual reference 
are also discussed. 


CHANGE FONT 


Introduction 

The Internet has enabled people to use various tools to ask for help. Visits by Americans to question and answer Websites 
have increased in recent years. Between 2006 and 2008, visits to social question and answer services increased by 889% 
( Hitwise, 20081 . The number of people who visited Yahoo! Answers, as measured by unique browser sessions, totaled over 
16.4 million in the month of September 2013 alone ( Quantcast, 2013) . 

In the last decade, library virtual reference has become increasingly commonplace in providing reference services to library 
patrons. Despite the popularity of virtual reference, national trends show that reference librarians are answering fewer and 
fewer questions every year ( Tvckoson, 201 1) . The Internet Public Library, which provides librarian-recommended Internet 
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resources and reference help to the public, has also received fewer questions in recent years ( IPL. 201 1) . This is backed by 
other reports of declining library reference transactions ( Kuhl, 2012) . 

Library communities speculate and debate over whether search engines, social question and answer services, Wikipedia, 
Facebook, and other social networks are replacing library reference services. Social question and answer services have 
increasingly sought to supplant the expertise of the reference librarian with the wisdom of the crowd ( Golbeck and 
Fleischmann, 2010 ). Some have even predicted the death of library reference services overall ( Henry, 2011 ; Pomerantz, 2010 ). 
However, others argue that reference services are not dead but have experienced a paradigm shift from physical to virtual 
reference services, and virtual reference needs to prove its relevance and value as a core, high-demand reference service 
(Young, 2013) . 

Problem statement 

Most previous research comparing virtual reference and social question and answer services has been conducted from the 
perspective of librarians, library users, or researchers. Studies from the viewpoint of social question and answer services users 
are sparse. A closer look at this user group and their comparison between using social question and answer services and virtual 
reference will offer additional insight regarding user behaviour in seeking information online, and important considerations for 
a virtual service. This line of research is also crucial to understanding social question and answer services users' awareness, 
use, and assessment of virtual reference, and to shed light on the relationship between the two services in meeting user 
information needs in the Internet environment. 

The study reported in this paper will address the research gap and identify areas for improvement within current virtual 
reference by exploring the following research questions: 

1. What questions do social question and answer services users ask while using the online service? 

2. How satisfied are users with answers received from social question and answer services? 

3. Are social question and answer services users aware of virtual reference and have they used the service? 

4. How do social question and answer services users evaluate and compare social question and answer services and virtual 
reference? 

5. What are the important considerations of a virtual service from a user's perspective? 

Literature review 

Virtual reference refers to all reference services provided online, whether synchronous or asynchronous (e-mail, LibAnswers, 
and Instant Message), or through SMS (text). Other nomenclatures include digital reference services, online reference services, 
and real-time services ( Nicholas, 2011 ). This literature review provides a synopsis of previous research involving virtual 
reference, its role in meeting user information needs, and its connection to social question and answer services. 

Change in user reference needs for virtual reference service 

Library virtual reference continues to evolve along with the changing technology and user expectations ( Kilzer, 2011 ). Today's 
users expect to get help where they are, minimising the overall effort associated with seeking information, and to identify the 
most convenient source to meet their information needs ( Chow and Croxton, 2012) . Although they may be capable of using a 
search engine to answer simple fact-based queries, users may still have difficulty sorting through the results of a research- 
based query, and this is where virtual reference might step in ( Numminen and Vakkari, 2009 : ( Solorzano, 2013) . 

In a longitudinal study of live chat and instant message reference services, Radford and Connaway (2013) observed a 
significant decrease in subject searching but an increase in procedural questions in the same time period. The authors 
suggested that the downturn in the subject reference types might be partially because of online tools such as search engines, 
Wikipedia, or other reference guides. The authors also suggested that changes in the makeup of virtual reference users could 
be another contributing factor. 

In responding to the evolution of various communication technologies, libraries have been offering virtual reference by 
telephone, e-mail and online chat to make the service as convenient as possible for users ( Luo, 2012) . The increasing 
availability and use of electronic resources allow virtual reference service providers to extend their reach and expand user 
horizons ( Agosto, Rozklis, MacDonald and Abels, 2011 ). 

Users' preferences and satisfaction with virtual reference 
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Research shows that users' preferences for virtual reference are dependent upon the type of question, the type of information 
seeker, and the convenience offered across library user groups ( Chow and Croxton, 2012) . Studies have found that 
convenience was considered an important reason for users to access virtual reference services, and that it was critical to library 
patrons' willingness to use virtual reference ( Connaway, Dickey and Radford, 2011 ; Nicol and Cook, 2013 ). 

One challenge for current virtual reference is its low usage by patrons, despite the fact that many studies established very high 
user satisfaction and inclination to recommend the service ( Pomerantz and Luo, 2006 ). The blame for the decrease in usage 
has been placed on the availability of virtual reference-like services on the Internet, users' overt confidence in their searching 
skills, and the library anxiety involved in talking to a librarian ( Connaway, Radford and Williams, 2009 1. Additionally, 
visibility of the service and the individual user's willingness to utilise the service are also important factors contributing to 
virtual reference use ( Mu, Dimitroff, Jordan and Burclaff, 2012) . 

Connaway and Radford (2011) suggest that user reassurance and positive user experience is key to address the virtual 
reference use challenge. They point out that without an explicit, live introduction to virtual reference, many users will not 
bring the trust and appreciation they feel for librarians into an online experience. It is possible for users and librarians to 
greatly value how they are treated in addition to the accomplishment of the encounter's goal to facilitate information discovery 

and use ( Radford, Radford. Connaway and DeAngelis, 2011) . 

Previous research has shown user perception and user satisfaction are very important in the virtual reference process. Such 
studies paid attention to the gap between virtual reference and library users' satisfaction. However, little is known about social 
question and answer services users' perception of virtual reference in their searching process (especially those users who use 
both) and what role the types of questions may play in this process. 

Socialisation of virtual reference 

In essence, virtual reference is an interactive process between librarians and users. Each interaction and query brings 
something new to the librarian and the user ( Tyckoson, 201 1 ). The effectiveness of social interaction, for example, in 
asynchronous e-mails, may depend on the interpersonal and effective communication that takes place between librarians and 
users ( Park, Li and Burger, 2010) . The tools to provide virtual reference will continue to change, but those key elements will 
remain the same—forming a connection with users and helping them find what they are looking for ( Solorzano, 2013) . A lack 
of social cues is seen as an aspect of chat reference to which librarians must adapt ( Gronemver and Deitering. 2009) . 

Online reference interactions present some unique challenges. Librarians who are part of a collaborative service may have 
difficulty answering questions about local policies at libraries in other parts of the state or country ( Bishop, Sachs-Silveira and 
Avet, 201 1) . For reference research, online social reference exemplifies a new stage that involves a transition from question 
negotiation to an online collaborative group effort. Unlike library virtual reference, social reference is conducted through 
cooperative effort and open participation to make obtaining reference help much more convenient ( Radford, 2008 ). The 
socialisation of virtual reference asks for virtual reference providers to embrace the change and challenge. 

Differences between social question and answer services and virtual reference 

At present, there is an increasing interest in understanding social question and answer services and virtual reference's 
similarities and differences, and most importantly the lessons that could be learned from one to enhance the other ( Gazan, 
2007 ; Kitzie and Shah, 2011 ). Additionally, research involving both social question and answer services and virtual reference 
suggests that it is important to understand how users view the two services with regard to meeting their reference needs ( Shah 
and Kitzie, 2012 ). 

In a recent study, Radford et al. (2013) found that users saw virtual reference services as authoritative and objective, more 
synchronous, and as receptive to more complex questions. Social question and answer services, however, were viewed as 
asynchronous, less authoritative, having simpler questions, and providing more opinionated answers. In terms of performance, 
Shah and Kitzie (2012) found that virtual reference outperforms social question and answer services in the following aspects: 
customisation, quality, relevance, accuracy, authoritativeness, and completeness. On the other hand, social question and 
answer services outperformed virtual reference in the following areas: cost, volume, speed, social aspects, engagement, and 
collaboration. Some additional differences between virtual reference and social question and answer services recognised by 
previous research include the dyadic reference encounter vs. teamwork, relevant boundaries vs. blurred boundaries, limit of 
authority vs. empowerment of authority, and institutional human and information resources vs. volunteers and free online 
information ( Shachaf. 2010) . 

It should be noted that most current studies that compare social question and answer services and virtual reference have 
focused on librarians and college students, who are more likely to be familiar with library virtual reference but not necessarily 
social question and answer services. This is shown in the study by Shah and Kitzie (2012) where the majority of the students 
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did not use social question and answer services to ask or answer a question. Studying social question and answer services users 
regarding their experience with social question and answer services and virtual reference can yield new insights when 
comparing the two services in serving user needs. Also, research samples of previous studies tended to be small due to their 
qualitative nature, with no more than forty participating subjects from one institution. The samples may not be representative 
of virtual reference and social question and answer services users, and findings may not be generalisable. 

Currently, an interesting avenue to explore is how virtual reference can complement social question and answer services to 
provide more comprehensive information when needed. Such collaboration may allow social question and answer services to 
offer premium (paid) content to their large user-base, and for virtual reference to create a new revenue stream to support 
sustainability ( Radford, Connaway and Shah, 2012) . Kitzie, Choi and Shah, (2012) conceptualised social question and answer 
services and virtual reference services under one umbrella, namely online question and answer services, and investigated both 
these services and virtual reference together. The findings indicated that social question and answer services and virtual 
reference services may in fact be directly comparable and, from learning concordant strengths and weaknesses, be improved as 
a whole. 

Methods 

Yahoo! Answers as the social question and answer services site 

For this study, Yahoo! Answers was chosen as the social question and answer services site for several reasons: 

• It is the top-ranking (Blogger Journey, 2013; Hitwise, 2008; McGee, 2008) social question and answer services service, 
with 54 million unique users in the U.S. and 250 million worldwide. Currently, Yahoo! Answers has approximately 300 
million resolved questions. 

• It has well-established procedures for questions and answers. The format of interaction on Yahoo! Answers is that a 
registered user posts a question in one of the pre-defined categories, and other users reply directly to that question with 
their answers. 

• It has a user management system offering incentives and rewards for positive contributions. 

• It uses a subject-based hierarchical category system to organise questions and answers. There are twenty-six categories 
in Yahoo! Answers. The wide variety of categories include: arts and humanities, education and reference, computers and 
Internet, and science and mathematics. 

• It allows sophisticated data extraction. Yahoo! Answers provides an application programming interface (API), letting 
users search for questions on any topic and by category, including subcategories 

Sample and data collection 

An online survey was conducted to understand social question and answer services uses and to seek users' input on their 
experiences and comparisons between virtual reference and social question and answer services. The survey sample consisted 
of 1,431 Yahoo! Answers users, who posted or answered a question in the Yahoo! Answers education and reference category 
during the time period of February 27 to March 30, 2013. The subject category was chosen because its theme is similar to 
virtual reference. 

The survey was hosted in Qualtrics , a secure online survey tool available to the researchers through institution subscription. 
The sample subjects were invited by the e-mail platform in Yahoo Answers! to participate in the survey between June 21 - 
July 15 with one follow-up in the middle. No incentive was offered for participation. The survey was designed to understand 
uses and comparisons between Yahoo! Answers and virtual reference, and the underlying motivations and concerns during the 
informational and social process. The survey questions, which contained multiple-choice, fill-in, and open-ended questions, 
covered the following areas with details about specific questions in the findings section where related results are reported: 

• service awareness of social question and answer services and virtual reference, 

• motivation to answer questions in social question and answer services, 

• uses and comparison of social question and answer services and virtual reference, 

• most recent experience using Yahoo! Answers and other sources to find answers to questions, 

• important considerations in using any online services to seek answers, and 

• basic demographics, related skills, and familiarity with various tools and services for information. 

Out of the 1,431 users in the sample, thirty could not be reached because of unavailable accounts at the time of the study. After 
one round of follow-up reminders, 230 responded to the online survey, with a response rate of 16.4%. The basic demographics 
of the respondents are summarised in Table 1. Among those who participated in the study, 58% were male and 42% were 
female; and 65% had a college or post-graduate degree. Their ages ranged from 18 to 82, with 18% being 18-19, 15% in their 
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20s, 10% in their 30s, 14% in their 40s, 20% in their 50s, 19% in their 60s, and 5% being 70 or older. The participants' Yahoo! 
Answers user levels tended to be high with 44% in Level 7, the highest user level that may be achieved for positive 
contributions, which is earned by answering questions, receiving best answer recognition as chosen by question-askers, and 
participating actively. Among remaining participants, there were 12% each in Level 6 and Level 5, with 14% and 11% for 
Level 4 and Level 3 respectively. New users start at Level 1, and most infrequent users remain in Level 1 and Level 2 if they 
do not consistently answer questions to earn positive points to move their user level higher. 


Table 1: Participants' profile (N=230). 


Category 

Count Percentage 

Male 

Sex (n= 192) 

111 

58% 

Female 

81 

42% 

18-19 

33 

18% 

20-29 

27 

15% 

30-39 

18 

10% 

Age (n= 186) 40-49 

26 

14% 

50-59 

37 

20% 

60-69 

36 

19% 

70 and above 

9 

5% 

Less than high school 

0 

0% 

High school 

36 

19% 

Education (n= 192) Some college 

32 

17% 

College 

72 

38% 

Post-graduate 

52 

27% 

Level 1 

3 

2% 

Level 2 

10 

6% 

Level 3 

19 

11% 

Yahoo! Answers user level (n=178) Level 4 

25 

14% 

Level 5 

22 

12% 

Level 6 

21 

12% 

Level 7 

78 

44% 


Limitations 

The survey response rate at 16.4% is considered low. This is a common challenge faced by these types of studies that directly 
survey social question and answer services' users, particularly those on Yahoo! Answers. All of the very few studies that 
surveyed Yahoo! Answers users have reported low response rates. For example, in Oh's (20111 study of health answerers' 
behaviour using the site, 257 responded out of 2,139 contacted, giving a response rate of 12.0%. In Dearman and Truong's 
(20101 study of why users of the social question and answer services site do not answer questions, the researchers contacted 
731 users and received 135 responses, yielding a rate of 18.5%. 

Several factors contribute to the low survey response rate of this type of study. First, there is a restriction of ten e-mails per 
account per day at Yahoo! Answers, which limits the number of participants reached by the surveyor. Secondly, the site has a 
feature that allows users to hide their account and e-mail contact information, which makes direct e-mail contact impossible. 
Thirdly, the site systematically eliminates accounts that are used for sending a large number of e-mails to other users. During 
the course of this study, several accounts used to reach the sample users were deactivated by Yahoo! Answers. Fourthly, the 
social question and answer services site's users tend to be very cautious and on high alert for spam content. In this study, a 
handful of e-mail inquiries and a few phone calls were received to verify if the survey invitation was indeed a real and valid 
request. Finally, a lack of incentive also contributed to the low response rate. A few e-mail inquiries asked for an incentive to 
participate. 

To fill the gap of previous research that focused on library users, the intended sample of this study was social question and 
answer services users, and participants were recruited within a social question and answer services environment. As social 
question and answer services users, they are likely to have more positive attitudes toward social question and answer services 
than the average person. This study tried to limit potential bias by only asking those who have used both social question and 
answer services and virtual reference to offer their comparison and evaluation of the two services. A stratified sampling of 
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virtual reference and social question and answer services users and non-users would be most desirable for future research to 
shed additional light for comparison. 

Given these limitations, more research with multiple user samples using this direct survey method and other methods such as 
interviews and focus groups is needed to generate a more complete picture of virtual reference and social question and answer 
services comparisons. 

Findings 

Type of questions asked at the social question and answer services site 

Two separate questions in the survey asked participants whether they had used the social question and answer services site to 
ask questions and to answer questions. Among those who participated in the study, most had both asked questions (85%) and 
answered questions (99%) at Yahoo! Answers. For those who indicated that they had asked a question, a follow-up question 
asked what types of questions they had asked in the past and to select all question types that applied, along with an option to 
enter other types if the provided types did not apply. Table 2 summarises the types of questions asked. The most common 
types of questions posted on the social question and answer services site are those asking for advice or opinion (60%), 
questions about specific facts (51%), and questions for leisure or entertainment (50%), with at least half of the participants 
having asked these types of questions. Questions about personal and/or private issues and for research are the next two 
common types of questions with about 40% of participants involved. Questions for school homework (25%) and for work 
(16%) are the two least common categories. 


Table 2: Types of questions asked in Yahoo! Answers (n=177). 


Type of question 

No. of Responses 

% 

Questions asking for advice or opinions 

106 

60% 

Questions about specific facts 

91 

51% 

For leisure or entertainment 

89 

50% 

About personal and/or private issues 

70 

40% 

For research 

68 

38% 

For school homework 

45 

25% 

For work 

28 

16% 

Other 

28 

16% 


Note. % does not total 100% as types of questions are not mutually exclusive and participants chose all that applied. 


Those participants who had asked questions had a follow-up question to recall the specific question asked most recently. The 
rank of common types of questions recently asked by participants (see Table 3) is in line with the common practices of askers 
on the site, which suggests that Yahoo! Answers users mostly engage in advice-seeking and fact-finding for leisure or personal 
issues; seldom using the site for work. 

Other types of questions reported include specific examples of questions people encounter in their daily lives, such as advice 
and help with technology related issues, questions about health, motorcycle repair, study options, sports, or questions to make 
people think and fine-tune ideas and to create social awareness. 


Table 3: Types of questions asked most recently in Yahoo! Answers (n=174). 


Type of question 

No. of Responses 

% 

1. Questions asking for advice or opinions 

76 

44% 

2. Questions about specific facts 

40 

23% 

3. For leisure or entertainment 

33 

19% 

4. About personal and/or private issues 

27 

16% 

5. For research 

25 

14% 

6. For school homework 

16 

9% 

7. For work 

12 

7% 

Other 

28 

16% 


Note. % does not total 100% as types of questions are not mutually exclusive and participants chose all that applied. 
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User satisfaction with answers received in social question and answer services 

Out of 179 participants who had asked questions using Yahoo! Answers, 172 answered a follow-up question in the survey 
questionnaire to rate their satisfaction with the specific question asked most recently by choosing one from: a) very satisfied, 
b) somewhat satisfied, c) neither satisfied nor dissatisfied, d) somewhat dissatisfied, and e) very dissatisfied. As reported in 
Table 4, 70% of the users were either very satisfied (31%) or somewhat satisfied (39%), 12% were neither satisfied nor 
dissatisfied, and 18% were somewhat dissatisfied (10%) or very dissatisfied (8%). It is interesting to note that 58% of the users 
did not seek any other sources after receiving the answer from the Yahoo! Answers site in their most recent encounter. A 
notable 26% of the users conducted Internet searching after receiving their answer. The major reasons cited for such post- 
receiving-answer activities were to verify the answer received (43%), to have more options (36%), and being unsatisfied with 
the answer received (30%). 


Table 4: User satisfaction with the answers 
received (n=172). 


Level of user satisfaction 

Responses 

% 

Very satisfied 

53 

31% 

Somewhat satisfied 

53 

39% 

Neither satisfied nor dissatisfied 

20 

12% 

Somewhat dissastisfied 

18 

10% 

Very dissatisfied 

14 

8% 

Total 

172 

100% 


It is noted that user satisfaction with answers received in social question and answer services varies by the type of question 
when the responses reported for Table 3 and Table 4 are cross examined. Table 5 summarises the calculated mean score of 
user satisfaction by question type. The mean score is calculated based on the scale: 1 for very satisfied, 2 for somewhat 
satisfied, 3 for neither satisfied nor dissatisfied, 4 for somewhat dissatisfied, and 5 for very dissatisfied. As some respondents 
indicated their asked question belongs to multiple question types, only those responses with one unique question type were 
selected for comparison. 

Overall, social question and answer services users are most satisfied with answers to their schoolwork questions, followed by 
questions about specific facts, and questions for advice or opinions. The least satisfied answers are those addressing research 
questions and about personal and/or private issues. A one-way ANOVA (analysis of variance) test result shows there is not a 
significant effect of question type on the level of user satisfaction with the answer received [F(4, 102) = 1.699, p = 0.156]. 


Table 5: User satisfaction with the answers received by question type. 


Rank 

Question type 

Mean 

Standard deviation (SD) 

1 

For school homework (n=8) 

1.63 

1.06 

2 

Questions about specific facts (n=15) 

2.07 

1.39 

3 

Questions asking for advice or opinions (n=46) 

2.15 

1.01 

4 

For work (n=4) 

2.25 

1.89 

5 

For leisure or entertainment (n=T9) 

2.26 

1.28 

6 

About personal and/or private issues (n=6) 

2.50 

1.05 

7 

For research (n=9) 

3.44 

1.33 

Note. 1 

= Very satisfied, 2 = Somewhat satisfied, 3 = Neither satisfied nor dissatisfied, 4 = 

Somewhat dissatisfied, 5 = Very 


dissatisfied. 

Virtual reference awareness and use 

Table 6 summarises virtual reference awareness and use among participants. The awareness and use of virtual reference is 
low. In this study, 68% of the users have never heard of virtual reference, 14% of them are aware of the service but have never 
used it, 13% use virtual reference only sometimes, and only 4% use virtual reference regularly. 


Table 6: Virtual reference use and awareness by Yahoo! Answers 
users (n=202). 
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Virtual reference awareness and use 

Responses 

% 

Never heard of it 

138 

68% 

I am aware of the service but have never used it 

28 

14% 

I use it only sometimes 

27 

13% 

I use it on a regular basis 

9 

4% 

Total 

202 

100% 


Those twenty-eight participants who were aware of the library virtual reference service but have never used it were asked 
about the main reason for not having used the service. The answers can be summarised in the following categories: 

• Have not needed the service. A majority of the respondents (64%) indicated that they have never felt the need to use a 
library virtual service. A few explained that they could find what they needed using search engines or Yahoo! Answers, 
or by directly consulting with people. Others specified that they may use virtual reference if they are in school, pursuing 
anything academic in nature, or needing something very comprehensive or esoteric. 

• Unfamiliar with it. About 18% of the respondents cited unfamiliarity with library virtual reference services as the reason 
why they have not used them. One did not know such services are free. 

• Too difficult or complicated/didn't know how to use. About 14% expressed that library virtual reference seems 'too 
complicated to use' and that they found other 'easier methods', such as an Internet search. 

• Negative past experience. Only one respondent (4%) cited unsuccessful previous experience using the service as the 
reason for not using it. 

User evaluation and comparison of social question and answer services and virtual reference 

A follow-up question in the survey asked respondents who have used both Yahoo! Answers and virtual reference (36 in total) 
to evaluate and compare the two services based on their experience. Twelve statements related to social question and answer 
services and virtual reference characteristics that are comparable, based on previous literature, were provided for the 
comparison and evaluation (see Table 7). For each statement, participants were asked to provide their assessment by choosing 
one of the following: Yahoo! Answers is much better, Yahoo! Answers is slightly better. About the same. Virtual reference is 
slightly better. Virtual reference is much better, or No opinion. 

Table 7 summarises the evaluations and comparisons by combining the two choices in favour of Yahoo! Answers and the two 
choices in favour of virtual reference. The results show that Yahoo! Answers stands out in the following areas over virtual 
reference: 

• advice-seeking and opinion questions (69% in favour of Yahoo! Answers, 6% in favour of virtual reference), 

• enjoyment (69% vs. 0%), 

• accessibility (49% vs. 14%), 

• promptness (46% vs. 24%), and 

• related experience (41% vs. 19%). 

On the other hand, virtual reference is considered better than Yahoo Answers in the following areas: 

• trust (75% vs. 6%), 

• quality (59% vs. 16%), 

• expertise (54% vs. 16%), and 

• ability to answer factual questions (49% vs. 11%). 

Although about half of the users consider both services to be about the same in the areas of user privacy (53%), being 
comfortable asking any questions (53%), and overall service satisfaction (47%), virtual reference is slightly stronger in user 
privacy (25% vs. 11%) while Yahoo! Answers has an edge in overall user satisfaction (33% vs. 17%) and making users feel 
comfortable asking any questions (28% vs. 11%). Overall, the strengths and weaknesses identified by the users who have used 
both social question and answer services and virtual reference are consistent with previous studies as reported in the literature 
review. 


Table 7: Evaluation and comparison of Yahoo! Answers and virtual reference. 

Yahoo! Answers is About the Virtual reference is 
better same better 

The service answers advice-seeking and opinion 
questions (n=36) 


No 

opinion 


69% 


19 % 


6% 


Total 


6 % 100 % 
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It's fun (n=36) 

69% 

Service is accessible 
(n=36) 

49% 

Prompt response (n=36) 

46% 

Answers are provided by people 
with related experience (n=36) 

41% 

My privacy is guarded 
(n=36) 

11% 

Overall, I am satisfied 
with the service (n=36) 

33% 

I feel comfortable asking 
any questions (n=36) 

28% 

Answers can be trusted (n=36) 

6% 

Answers are of good quality 
(n=36) 

16% 

Answers are provided by 
people with expertise (n=36) 

16% 

The service answers factual 
questions (n=36) 

11% 

Note. Bold numbers indicate the highest response for each evali 


25% 

0% 

6% 

100% 

35% 

14% 

3% 

100% 

27% 

24% 

3% 

100% 

35% 

19% 

5% 

100% 

53% 

25% 

11% 

100% 

47% 

17% 

3% 

100% 

53% 

11% 

OO 

o x 

100% 

17% 

75% 

3% 

100% 

22% 

59% 

3% 

100% 

27% 

54% 

3% 

100% 

37% 

49% 

3% 

100% 


item. 


Considerations of a virtual service 


Social question and answer service users in this study were asked to rate the importance of various considerations when they 
used any online or virtual service to seek answers to their questions. Besides the comparable statements used for social 
question and answer services and virtual reference comparisons, additional statements about unique features of one or the 
other were also provided for rating; these included the number of responses for a given question, opportunity for friendship or 
connection with people, a variety of responses and opinions for a question, and answers by people directly and specifically 
instead of found by a search engine were added for virtual service considerations. In addition, there was an other category that 
allowed participants to enter and rate their own key consideration for a virtual service. 

For each statement, participants were asked to provide their rating by choosing one from: a) very important, b) important, c) 
neither unimportant nor important, d) unimportant, e) very unimportant, and f) not applicable. 

Table 8 summarises the importance ratings and their calculated mean scores and standard deviations using the scale with 1 for 
very important, 2 for important, 3 for neither important nor unimportant, 4 for unimportant, and 5 for very unimportant; not 
applicable ratings were not included in the calculations. As shown in Table 8, the following considerations, with a mean score 
of better than important (2), are the top considerations for using an online or virtual question and answer service: 

1. quality of answers (1.40) 

2. accessibility of service (1.41) 

3. trust (1.46) 

4. promptness (1.57) 

5. related experience (1.60) 

6. expertise (1.64) 

7. privacy (1.75) 

8. feeling comfortable asking questions (1.88) 

9. other (1.89) 

The other important considerations suggested by users included cost, ability to evaluate results, continued learning/awareness, 
personal/sincere aspect, accuracy, and no jargon when using the service. 

To determine the underlying latent clusters within the thirteen specific user considerations, we conducted principal 
components analysis with varimax rotation. The other consideration was not included as it represents a small but different set 
of considerations. Factors were retained based on Eigenvalues greater than 1. Items were assigned to factors based on their 
largest loading. We also calculated the Cronbach's Alpha coefficient for each factor to determine its scale reliability. As shown 
in Table 9, the analysis yielded the following four factors explaining a total of 66.25% of the variance for the entire set of 
variables: 
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• Factor 1 is labelled human answer because of the high loadings of the following three considerations related to people 
who answer the questions: answers are from people with related experience, from people with expertise, and from 
people instead of a search. This first factor explains 19.60% of the variance. 

• Factor 2 is labeled answer sen’ice, which has the high loadings in the following four answer service-related 
considerations: promptness, accessibility, trust, and quality of answers received. This factor explained 17.62% of the 
variance. 

• Factor 3 is labeled active response experience, which has high loadings in the following three considerations related to 
an active response service: number and variety of responses, opinions for a given question, and enjoyment. This factor 
explained 17.31% of the variance. 

• Factor 4 is labeled personal social concern, which has high loadings in the following three considerations that relate to 
people's personal concerns while interacting socially: feeling comfortable asking questions, privacy, and opportunity for 
friendship/connection with other people. This factor explained 11.73% of the variance. 

The communalities of the variables included are all above 0.50, six of which are above 0.70. This result means that the 
extracted factors explain most of the variance in each of the variables. Cronbach's Alpha for the four factors overall and the 
first three factors are over 0.72, suggesting a relatively high internal consistency of the items/considerations being related as a 
group. The fourth factor's Cronbach's Alpha is below 0.7 and this factor needs to be interpreted with caution. 


Table 8: Considerations and their importance ratings for online services. 


Consideration 


1. Quality of answers (n=l 82) 

2. Service is accessible (n=182) 

3. Answers can be trusted (n=179) 

4. Prompt response (n=l81) 

5. Answers are provided by 

people with related experience (n=179) 

6. Answers are provided by 
people with expertise (n=179) 

7. My privacy is guarded (n=l81) 

8.1 feel comfortable asking 
any questions (n=180) 

9. Other (please specify) (n=28) 

10. Answers are provided by 
people directly and specifically 
instead of found by a search engine 
(n=181) 

11. There are a variety of 
responses and opinions (n=184) 

12. Number of responses for 
a given question (n=181) 

13. It's fun (n=181) 

14. Opportunity for friendship 

or connection with people (n=172) 




Neither 

Very 

important 

Important 

important 

nor 



unimportant 

117 

57 

8 

(64%) 

(31%) 

(4%) 

116 

59 

6 

(64%) 

(32%) 

(3%) 

114 

50 

13 

(64%) 

(28%) 

(7%) 

94 

74 

10 

(52%) 

(41%) 

(6%) 

94 

65 

17 

(53%) 

(36%) 

(9%) 

93 

62 

20 

(52%) 

(35%) 

(11%) 

95 

50 

25 

(52%) 

(28%) 

(14%) 

76 

61 

34 

(42%) 

(34%) 

(19%) 

14 

4 

9 

(50%) 

(14%) 

(32%) 

56 

66 

45 

(31%) 

(36%) 

(25%) 

42 

80 

40 

(23%) 

(43%) 

(22%) 

35 

63 

67 

(19%) 

(35%) 

(37%) 

46 

52 

50 

(25%) 

(29%) 

(28%) 

23 

26 

60 

(13%) 

(15%) 

(35%) 


importai 

lt Very 

unimportant 

Standard 
Mean Deviation 
(SD) 

0 

0 



(0%) 

(0%) 

1.40 

0.574 

0 

1 



(0%) 

(1%) 

1.41 

0.613 

2 

0 



(1%) 

(0%) 

1.46 

0.681 

2 

1 



(1%) 

(1%) 

1.57 

0.700 

3 

0 



(2%) 

(0%) 

1.60 

0.730 

3 

1 



(2%) 

(1%) 

1.64 

0.790 

9 

2 



(5%) 

(1%) 

1.75 

0.950 

7 

2 



(4%) 

(1%) 

1.88 

0.925 

1 

0 



(4%) 

(0%) 

1.89 

0.994 

9 

5 



(5%) 

(3%) 

2.12 

0.998 

20 

2 



(11%) 

(1%) 

2.24 

0.962 

13 

3 



(7%) 

(2%) 

2.37 

0.932 

21 

12 



(12%) 

(7%) 

2.45 

1.181 

33 

30 



(19%) 

(17%) 

3.12 

1.253 


Table 9: Communalities and factor loadings for considerations of virtual/online services. 
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Loadings 


Consideration 

Factor 1: 
Human 

answer 

Factor 2: 
Answer 
service 

Factor 3: 
Active 
response 

Factor 4: 
Personal 
social 

Commun- 

ality 

Total/ 

overall 


experience 

concern 



Answers are from people with related experience 

0.809 




0.747 


Answers are from people with expertise 

0.769 




0.725 


Answers are from people instead of a search 

0.759 




0.602 


Prompt response 


0.801 



0.721 


Service is accessible 


0.793 



0.645 


Answers can be trusted 


0.632 



0.607 


Quality of answers 

0.435 

0.550 



0.506 


A variety of responses and opinions 



0.838 


0.749 


Number of responses for a given question 



0.805 


0.741 


It's fun 



0.696 

0.427 

0.701 


I feel comfortable asking 
any questions 




0.686 

0.626 


My privacy is guarded 




0.668 

0.633 


Opportunity for friendship/connection 
with people 



0.460 

0.574 

0.608 


Eigenvalue 

2.547 

2.290 

2.250 

1.525 



% of total variance explained 

19.60% 

17.62% 

17.31% 

11.73% 


66.25% 

Cronbach's Alpha 

0.784 

(k=3) 

0.721 

(k=4) 

0.732 (k=3) 

0.469 

(k=3) 


0.773 

(k=13) 


Note. Factor loadings <0.4 are suppressed. 


Table 10 shows the mapping of the top eight most important considerations for virtual online services from Table 8, with the 
results of user evaluation and comparison of virtual reference and Yahoo! Answers as summarised in Table 7. This mapping 
and comparison of virtual reference and social question and answer services shows where the two services stand in user 
evaluations for the top eight most important considerations when using online services in general. It is interesting to note that 
these key considerations are evenly shared between virtual reference and Yahoo! Answers; neither service stands out in all of 
them. This result suggests that each platform performs well in some areas valued most by users, while having room to learn or 
improve to meet users' needs and requirements/expectations for online services. 


Table 10: Comparison of virtual reference and social question and answer services on top 

eight important considerations. 

Consideration Better performer as evaluated by users 


1. Quality of answers 

2. Service is accessible 

3. Answers can be trusted 

4. Prompt response 

5. Answers are provided by 
people with related experience 

6. Answers are provided by people with expertise 

7. My privacy is guarded 

8. Feeling comfortable asking questions 


Virtual reference 
Yahoo! Answers 
Virtual reference 
Yahoo! Answers 

Yahoo! Answers 

Virtual reference 
Virtual reference 
Yahoo! Answers 


Discussion 


The objective of this study was to address the gap in user evaluation and comparison of social question and answer services 
and virtual reference in previous research. The results indicate that social question and answer services and virtual reference 
play different roles in meeting user information needs, and have their share of strengths and weaknesses individually as virtual 
services. 
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This study revealed that the types of questions asked on the social question and answer services site are mostly associated with 
users' daily lives, with most questions pertaining to advice or opinion and about half for leisure or entertainment. These results 
are consistent with findings by previous studies (e.g. Kim, Oh and Oh, 20071 , which indicate that conversational questions 
seeking opinions or suggestions are asked more often than informational questions on Yahoo! Answers, with users seldom 
using the site for work. Unlike social question and answer services, virtual reference tends to elicit a high number of questions 
that are fact-based, informational, subject-related, and research-orientated, rather than those that focus on personal opinions or 
thoughts ( Shah, Radford, Connaway, Choi, and Kitzie, 2012j . Overall, while both social question and answer services and 
virtual reference have been utilised for varying purposes and queries, each has been used more specifically for certain types of 
questions. Users decide between social question and answer services and virtual reference based on the types of questions they 
have at hand. 

Findings from this study concerning user satisfaction with social question and answer services and user comparisons between 
social question and answer services and virtual reference help explain user service choices and preferences. The study revealed 
that 70% of users are either very satisfied or somewhat satisfied with the answers they received from the social question and 
answer services site. This high overall user satisfaction with social question and answer services is consistent with findings 
from previous social question and answer services studies and comparable to virtual reference services (e.g., Chan, Ly and 
Meulemans, 2012 ; Connaway and Radford, 2011 ; Pomerantz and Luo, 2006 : Shah, 2011 ; Shah and Kitzie, 2012 ). 

Additionally, this study found that user satisfaction may vary by types of question. Users are more likely to be satisfied with 
answers to their schoolwork questions, questions about specific facts, and questions for advice or opinions. The questions 
eliciting the least satisfactory responses are those addressing research questions and personal/private issues. These results 
show the strengths and weaknesses of social question and answer services service in answering different types of questions, 
and help explain why users turn to virtual reference for research or difficult questions, as found in previous studies (e.g. 
Numminen and Vakkari, 2009j . 

This study found that social question and answer services users have low awareness and usage of virtual reference, with 68% 
having never heard of virtual reference, 14% being aware of the service but never utilising it, 13% using virtual reference only 
sometimes, and only 4% using virtual reference regularly. These findings add to previous virtual reference research and 
discussion about low virtual reference awareness and use in general ( Connaway and Radford, 2011 ). Attention must also be 
paid to those individuals who report awareness of virtual reference but have not yet used the service, citing the barriers of 
unfamiliarity and perceived difficulty. While reference librarians and their work are positively valued overall, social question 
and answer services users remain largely unaware of and are not familiar with reference services. These findings point to user 
awareness and service promotion as areas that libraries need to address in virtual reference. 

Recent studies show that many users did not seem to recognise the difference between the functions of social question and 
answer services and virtual reference ( Kitzie and Shah, 2011 ), and college students especially were less likely to be aware of 
virtual reference ( Gazan, 2007) . The lack of awareness of virtual reference found in this study may impede this group of users 
from taking advantage of the service. However, those who had used both social question and answer services and virtual 
reference showed a clear understanding of the differences between the two in their evaluations and comparisons, further 
confirming the different roles of the two services in meeting user information needs. 

This study has identified the important considerations and the underlying latent clusters of the considerations of an online or 
virtual information service. The results show that service accessibility, quality of answers, and credibility of answers are most 
important considerations, followed by promptness, answerers' expertise and related experience, and privacy. These results 
confirm the findings by Connaway, Lanclos, and Hood (2013) that indicate access is the key to meeting users' expectations, 
depending upon the context and situation of their needs. These results also confirm previous studies that convenience, speed, 
and efficiency are critical attributes for users of virtual services in their searches for information ( Nicholas, 2011 ; Nicol and 
Crook, 2013) . The considerations can be explained by four underlying factors: human answer (questions are answered by 
humans and by those with expertise and related experience), answer service (promptness, accessibility, trust, and quality), 
active response experience (number and variety of answers and fun experience), and personal social concern (feeling 
comfortable asking questions, privacy, and friendship). This result suggests people still value human-provided answers despite 
the availability of various search tools and resources. At the same time, quality service, user experience, and personal/social 
concerns are also valued. 

When mapping the important considerations for an online or virtual service with user evaluations of virtual reference and 
social question and answer services, it is clear that neither service is scored higher in every aspect. While virtual reference 
outperforms social question and answer services in quality of answers, tarstworthiness of answers, expertise of the people 
providing the answers, and privacy, social question and answer services performs better for the accessibility of the service, 
promptness, the related experience of those providing answers, and feeling comfortable when asking questions. The 
weaknesses of virtual reference services pinpoint areas where enhancements could be made to meet user needs. 
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Although privacy has been a core concern among librarians and a reason why some social media aspects have not been 
adopted in virtual reference ( Litwin, 2006 ), it is surprising that those who have used both virtual reference and social question 
and answer services do not see a major difference in privacy between the two services (as shown in Table 7), and privacy is 
ranked seventeenth among the important considerations for an online service (see Table 8) among all respondents of this 
study. This sharp difference in viewpoint in privacy issues needs to be further explored. 

Conclusions and suggestions for further research 

This study contributes to the ongoing discussion about the future of library reference services and the changes they must make 
to meet the increasing online infonnation environment. Based on user evaluation and comparison of social question and 
answer services and virtual reference, it is clear that each currently has its own strengths and weaknesses in serving various 
user needs, and in meeting all the important criteria that users seek in an online or virtual infonnation service. The notion that 
social question and answer services will take over virtual reference is not supported. 

Virtual reference and social question and answer services have great potential to collaborate in meeting user needs in the 
Internet environment. We have already seen some cases of virtual reference and social question and answer services 
integration. For example, many reference librarians are participating in Yahoo! Answers and offering quality reference 
services within this platform. Their involvement and services have received recognition by the Yahoo! Answers user 
community ( John, 2010 ). Another example is Radical Reference ( http://radicalreference.info/ ~), where a collective of volunteer 
library workers provide online reference services to activists, journalists, and researchers. The engagement and contribution of 
reference librarians provide good marketing and advertising opportunities for libraries and library services. On the other hand, 
while libraries have been widely adopting social media and have recognised the values of such platforms for improving virtual 
reference services ( Arya and Mishra, 201 1J , there has been little integration of social media and reference services beyond 
interrelation of the two for service promotion ( Benn and McLoughlin, 2013) . There has been ongoing effort in exploring how 
virtual reference services may work alongside social question and answer services and how virtual reference librarians may 
collaborate in forming a community with experts outside libraries for possible new service designs ( Radford et al, 2013 J . 

Although the specific model and infrastructure for future virtual reference services remains to be developed and tested, the 
future direction of virtual reference is clear with regard to meeting user expectations of virtual services in the online 
environment: quality, accessibility, trust, prompt response, expertise, user community, privacy, and openness to various 
questions. Libraries need to reach out to potential users who may not have ever asked a reference question or even know that 
libraries offer interactive virtual reference services. Additionally, the integration of Yahoo! Answers with Google searches 
provides a good example for potential collaboration between virtual reference and similar services, as its questions and 
answers are visible within Google results. For virtual reference, this could provide the opportunity to promote user awareness 
of the service with quality answers provided by librarians. 

This study confirms that virtual reference and social question and answer services can be chosen to meet different needs and 
serve different purposes. In addition, it identifies the important considerations for an online or virtual service. Future research 
should examine whether an online or virtual service is preferred by certain groups of users, and explore the reasons for this 
preference to gain a better understanding of users' information-seeking process. For example, are there users who prefer a 
service irrespective of their infonnation needs and situations? What are the user profiles of those who only use social question 
and answer services, but not virtual reference? What are the user profiles of those who only use virtual reference, but not 
social question and answer services? What is their infonnation-seeking path? Do they utilise other sources before and after 
using social question and answer services and/or virtual reference? Additionally, some questions in shaping future virtual 
reference operation also require exploration. For example, what are the effective ways for virtual reference to collaborate with 
social question and answer services? What are effective ways to direct questions to people with the appropriate expertise? 

How can some social elements of social question and answer services be integrated into virtual reference? What is the impact 
of social question and answer services-like technology on virtual reference? The answers to these questions will prepare 
virtual reference to better position itself to satisfy users' information needs more effectively as part of their infonnation- 
seeking process. 
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